The present disclosure relates to the field of image capture, and, in particular, to a color filter array and a method for performing color interpolation, also known as demosaicking.
Digital image acquisition devices, such as digital cameras, utilize optical sensors to capture images. The optical sensing elements are typically multi-pixel arrays of charge coupled devices (CCD) or complementary metal oxide semiconductors (CMOS). Both of these sensor types are inherently monochromatic. The incident radiation at each pixel of a sensor-array is integrated over a range of wavelengths (in which the device is sensitive) to give intensity values over the sensor-array. To acquire color images, cameras utilize filters that are sensitive in particular ranges in the visible spectrum that are placed before the optical sensor-array in the imaging pipeline. The sensor-array output is then the image band corresponding to the spectral transmission of the color filter. Since at least three color bands are required to represent an image for the human visual system, at least three sensor-arrays with three different color filters are required to acquire a color image. Although other color combinations are used, arrangements of color filters typically extract the three primary colors: red, green, and blue.
Such multi-sensor acquisition schemes have several drawbacks. For example, multi-sensor cameras typically include one or more beam-splitters that send the light to the different color sensors which contribute substantially to the cost of the camera. Also, since the color bands are acquired at different planes, a post-processing operation is required to correct for the associated misregistration. To avoid the cost and complexity of multi-sensor acquisition systems, most consumer-level digital color cameras employ only one optical sensor. The sensor is overlaid with a color filter array (CFA) such that only one color is sampled at each pixel location. The full-color image is reconstructed from the sub-sampled data in a later step commonly referred to as demosaicking. Demosaicking depends on the pattern that defines the layout of the filters on the pixels of the sensor.
The most significant desirable feature of a CFA pattern, particularly for devices that have limited computational capabilities (cell-phone cameras, low-end digital still cameras, PDA cameras, etc.), is the ease of demosaicking. Regular, repeated CFA patterns work best to satisfy this requirement. Techniques for performing demosaicking, sometimes referred to as “interpolation,” are known in the art. For example, U.S. Pat. No. 4,642,678 to Cok, U.S. Pat. No. 5,373,322 to Laroche et al., and U.S. Pat. No. 5,475,769 to Wober et al. describe various methods for recovering missing pixel values from sampled color image data. The Wober patent describes a common approach using a convolution kernel to compute missing pixel values for each pixel location based on the pixel values of a neighborhood of pixels surrounding each pixel location. For an n×m neighborhood of pixels, the convolution kernel is an n×m set of coefficients.
Another useful feature of uniform CFA patterns is their relative immunity to optical and electrical cross-talk among pixels in the sensor array. Cross-talk or leakage between adjoining differently colored pixels can significantly alter the effective spectral transmittance function of a pixel. Regular patterns ensure a measure of consistency in the transmittances of similarly colored pixels across a sensor-array. A drawback of regular arrays is that they may suffer from Moire artifacts, or beats, in cases where the scene has periodic patterns similar in frequency to the period of the CFA pattern.
A successful CFA pattern must adhere to the properties of the human visual system (HVS). One common periodic pattern is known as the Bayer pattern. In a Bayer pattern, green (50%) is sampled at twice the density of either red (25%) or blue (25%) since the luminance response of the human visual system (HVS) corresponds closely with the HVS response to the green range of the spectrum.
The most distinctive features of the HVS are its low-pass response to luminance and chrominance components, and the fact that the cut-off for the luminance component is between two and three times as high as the cut-off for the chrominance components. These properties have been used to arrive at the ideal NTSC human visual R:G:B ratios of 30:59:11. These ratios are difficult to achieve with periodic CFAs formed by repeating 2×2 patterns like the Bayer array.
It is desirable to provide for a CFA with a suitable arrangement of color filters that closely adheres to the properties of the HVS, and also addresses the above-described factors that affect CFA performance.